9 research outputs found
Nonlinear System Identification via Tensor Completion
Function approximation from input and output data pairs constitutes a
fundamental problem in supervised learning. Deep neural networks are currently
the most popular method for learning to mimic the input-output relationship of
a general nonlinear system, as they have proven to be very effective in
approximating complex highly nonlinear functions. In this work, we show that
identifying a general nonlinear function from
input-output examples can be formulated as a tensor completion problem and
under certain conditions provably correct nonlinear system identification is
possible. Specifically, we model the interactions between the input
variables and the scalar output of a system by a single -way tensor, and
setup a weighted low-rank tensor completion problem with smoothness
regularization which we tackle using a block coordinate descent algorithm. We
extend our method to the multi-output setting and the case of partially
observed data, which cannot be readily handled by neural networks. Finally, we
demonstrate the effectiveness of the approach using several regression tasks
including some standard benchmarks and a challenging student grade prediction
task.Comment: AAAI 202
Low-rank Characteristic Tensor Density Estimation Part II: Compression and Latent Density Estimation
Learning generative probabilistic models is a core problem in machine
learning, which presents significant challenges due to the curse of
dimensionality. This paper proposes a joint dimensionality reduction and
non-parametric density estimation framework, using a novel estimator that can
explicitly capture the underlying distribution of appropriate reduced-dimension
representations of the input data. The idea is to jointly design a nonlinear
dimensionality reducing auto-encoder to model the training data in terms of a
parsimonious set of latent random variables, and learn a canonical low-rank
tensor model of the joint distribution of the latent variables in the Fourier
domain. The proposed latent density model is non-parametric and universal, as
opposed to the predefined prior that is assumed in variational auto-encoders.
Joint optimization of the auto-encoder and the latent density estimator is
pursued via a formulation which learns both by minimizing a combination of the
negative log-likelihood in the latent domain and the auto-encoder
reconstruction loss. We demonstrate that the proposed model achieves very
promising results on toy, tabular, and image datasets on regression tasks,
sampling, and anomaly detection
Information-theoretic Feature Selection via Tensor Decomposition and Submodularity
Feature selection by maximizing high-order mutual information between the
selected feature vector and a target variable is the gold standard in terms of
selecting the best subset of relevant features that maximizes the performance
of prediction models. However, such an approach typically requires knowledge of
the multivariate probability distribution of all features and the target, and
involves a challenging combinatorial optimization problem. Recent work has
shown that any joint Probability Mass Function (PMF) can be represented as a
naive Bayes model, via Canonical Polyadic (tensor rank) Decomposition. In this
paper, we introduce a low-rank tensor model of the joint PMF of all variables
and indirect targeting as a way of mitigating complexity and maximizing the
classification performance for a given number of features. Through low-rank
modeling of the joint PMF, it is possible to circumvent the curse of
dimensionality by learning principal components of the joint distribution. By
indirectly aiming to predict the latent variable of the naive Bayes model
instead of the original target variable, it is possible to formulate the
feature selection problem as maximization of a monotone submodular function
subject to a cardinality constraint - which can be tackled using a greedy
algorithm that comes with performance guarantees. Numerical experiments with
several standard datasets suggest that the proposed approach compares favorably
to the state-of-art for this important problem
STELAR: Spatio-temporal Tensor Factorization with Latent Epidemiological Regularization
Accurate prediction of the transmission of epidemic diseases such as COVID-19
is crucial for implementing effective mitigation measures. In this work, we
develop a tensor method to predict the evolution of epidemic trends for many
regions simultaneously. We construct a 3-way spatio-temporal tensor (location,
attribute, time) of case counts and propose a nonnegative tensor factorization
with latent epidemiological model regularization named STELAR. Unlike standard
tensor factorization methods which cannot predict slabs ahead, STELAR enables
long-term prediction by incorporating latent temporal regularization through a
system of discrete-time difference equations of a widely adopted
epidemiological model. We use latent instead of location/attribute-level
epidemiological dynamics to capture common epidemic profile sub-types and
improve collaborative learning and prediction. We conduct experiments using
both county- and state-level COVID-19 data and show that our model can identify
interesting latent patterns of the epidemic. Finally, we evaluate the
predictive ability of our method and show superior performance compared to the
baselines, achieving up to 21% lower root mean square error and 25% lower mean
absolute error for county-level prediction.Comment: AAAI 202
Analysis and Utilization of Entrainment on Acoustic and Emotion Features in User-agent Dialogue
Entrainment is the phenomenon by which an interlocutor adapts their speaking
style to align with their partner in conversations. It has been found in
different dimensions as acoustic, prosodic, lexical or syntactic. In this work,
we explore and utilize the entrainment phenomenon to improve spoken dialogue
systems for voice assistants. We first examine the existence of the entrainment
phenomenon in human-to-human dialogues in respect to acoustic feature and then
extend the analysis to emotion features. The analysis results show strong
evidence of entrainment in terms of both acoustic and emotion features. Based
on this findings, we implement two entrainment policies and assess if the
integration of entrainment principle into a Text-to-Speech (TTS) system
improves the synthesis performance and the user experience. It is found that
the integration of the entrainment principle into a TTS system brings
performance improvement when considering acoustic features, while no obvious
improvement is observed when considering emotion features.Comment: This version has been removed by arXiv administrators because the
submitter did not have the right to assign a license at the time of
submissio